Since the year 2003 the police department of san francisco has been reporting crime data. All these data are avaible to the public and therefore relevant to do data analysis on. Of particular interest for data analysis is the different crime types, the time of incident (both date but also time of day down to the minute) but also the coordinates of the incident (given in latitude/longitude). From this data its possible to look at temporal and spatial trends of different crimes over the last 20+ years. The different categories of crimes include vehicle theft, vandalism, robbery, prostiution and many more.
We have chosen to look at the trends of vehicle thefts since the trend are “unique” compared to the other types of crimes. This will be shown in the following plots.
The temporal trend of vehicle thefts
The first relevant plot is the number of incidents of vehicle theft per year. We ignore 2025 since the we do not have data for that entire year.
Code
import pandas as pd import numpy as npimport matplotlib.pyplot as pltdata=pd.read_csv("C:/NoterDTU/6_semester/Social_data/website_2/s224394.github.io/merged_data.csv")crimes = data[['Category', 'Year']]crimes = crimes[(crimes['Category']=='VEHICLE THEFT') & (crimes['Year']!=2025) ]crime_counts = crimes["Year"].value_counts().sort_index()plt.figure().set_figwidth(7.4)crime_counts.plot(kind="bar",color="indigo",edgecolor="black")ax=plt.gca()ax.set_facecolor("#f5f5f5")plt.ylabel("Number of incidents")plt.xlabel("Year")plt.title("Number of Vehicle thefts per year (2003-2024)")plt.show()
One thing that seems unique is the sudden drop from 2005 to 2006 and onwards. In 2005 the numbers peak at around 17.500 vehicle thefts while the next year it drops by around 10.000 and remains in that range going forward. This is approximately 60% of the crimes that just stopped happening in one year. One would suspect that the police force might have increased the resoruces for fighting vehicle theft. One explanation could also be that cars generally are harder to break into today.
In the news article “Car Thefts Decrease Statewide” by east bay times (Staff 2007) also tells this story Where the general trend for vehicle theft are on the decline. The reason behind this trend is both the fact that more and more vehicle have implemented alarms, key-coding systems. But also there has also been set up 16 auto-theft task forces. There have also been an increase in the use of so called “bait-cars” which are used as bait to track down the drivers and since its normal that they steal more than one car the number of cars that are being stolen drops significantly. In 2006 they made 357 arrest with the use of bait-cars. Which might have severely impacted the amount of cars stolen.
The spatial trends
One thing one might suspect could explain the sudden drop is if there suddenly where a focus from the police deparment on a specific area. As previously mentioned they have the data for where the different crimes are happening and one would then think that they then priotize forces in these specific areas. In order to look at the we plot a time series of where the incidents of vehecle theft is happening for a given month and try to see if there is any difference.
Code
import foliumfrom folium.plugins import HeatMapWithTimefrom IPython.display import display# Load datadf = pd.read_csv("C:/NoterDTU/6_semester/Social_data/website_2/s224394.github.io/merged_data.csv")# Filter for vehicle thefts between 2003-2007df_filtered = df[(df['Category'] =='VEHICLE THEFT') & (df['Year'].between(2003, 2024))].copy()# Extract relevant columns and drop NAdf_filtered = df_filtered[['Latitude', 'Longitude', 'Month', 'Year']].dropna()# Check for valid coordinatesvalid_coords = df_filtered[ (df_filtered['Latitude'].between(-90, 90)) & (df_filtered['Longitude'].between(-180, 180))]# Define month mapping and ordermonth_mapping = {"January": 1, "February": 2, "March": 3, "April": 4, "May": 5, "June": 6, "July": 7, "August": 8, "September": 9, "October": 10, "November": 11, "December": 12}month_names =list(month_mapping.keys())# Create numerical month columndf_filtered['MonthNum'] = df_filtered['Month'].map(month_mapping)# Sort by year and monthdf_filtered = df_filtered.sort_values(['Year', 'MonthNum'])# Prepare heat data and time indexheat_data = []time_index = []for year inrange(2003, 2025):for month_num inrange(1, 13): month_data = df_filtered[ (df_filtered['Year'] == year) & (df_filtered['MonthNum'] == month_num) ] coords = month_data[['Latitude', 'Longitude']].values.tolist() heat_data.append(coords) time_index.append(f"{month_names[month_num-1]}{year}")# Only create map if we have data# Create base mapbase_map = folium.Map(location=[37.75800, -122.41914], zoom_start=11.5,zoom_control=0,scrollWheelZoom=False,dragging=0)# Add heatmap with timeHeatMapWithTime( heat_data, index=time_index, # Time labels showing month and year auto_play=0, max_opacity=0.5, radius=11, min_opacity=0.1, gradient={0.2: 'blue', 0.4: 'lime', 0.6: 'orange', 0.8: 'red'}, display_index=True, use_local_extrema=1, name="Vehicle Thefts", blur=1).add_to(base_map)# Display mapdisplay(base_map)
Make this Notebook Trusted to load map: File -> Trust Notebook
From the heatmap its clear that a lot of the incidents happen in the north-eastern/eastern part of san francisco. And that trend does not change as teh years go by. But its clear that there is fewer crimes compared to 2003-2005 and the years after since the point of the heat map are more spread out. 😎 Hello is this thing working?
After 2006 the number of incidents are generally low compared to there is still two times an upward trend. After 2010 and after 2019. Might inital guess would be that they are both related to the econimic crisis that happende around there. The 2009 economic crahs and the begenning of the corona pandemic both made it so more people were poor and therefore needed to steal.
Last thing that could be interesting to look at is if the trends in data are unique to car theft or if the over all trend of crime is the “same” as with the car theft.
Correlation between crimes
In order to compare the crimes. We choose to look at how correlated the different crimes are. What we are comparing is the amount of crimes for a given month example burglary versus vehicle theft in the month of january 2015. We can the make an scatter plot and compute how related the data are. The scatter plot might also show other trends. But we will get to that. 😎
When going through the different options and comparing them. One notices that there is close to zero correlation between any of the crimes and vehicle theft. Where as the big reason behind it is the incidents of the years 2003-2005 which are completely different for vehicle theft. If you look at an example as robbery and assault they are way more correlated (\(r^2\) value of 0.485) and generally if we compare most of the crimes with the total number of crimes, then they are fairly correlated. Examples being vandalism having \(r^2=0.486\) and larcency/theft having \(r^2=0.700\). Some of this might also be explained in larcency/theft and vandalism playing a bigger part of the crime incidents numbers
One interesting thing one could look at is if not including the years 2003-2005 how much that would impact it.
